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Across the U.S., youth development approaches are being 
tested in out-of-school time programs as a strategy to 
combat the growing opportunity gap between privileged 
and underprivileged youth (Gardner, Roth, & Brooks-Gunn, 
2009). Along with increased recognition of the value of 
youth development programming has come increased 
financial support (Padgette, 2003; Zeller-Berkman, 2010). 

This investment, in turn, brings increased pressure to 
continually prove to funders that youth development pro¬ 
grams affect student outcomes (Zeller-Berkman, 2010). 
The increased emphasis on accountability has sometimes 
forced community-based organizations (CBOs) to main¬ 
tain a myopic focus on outcomes that are easily measur¬ 
able but not necessarily the most important (Fusco, Law¬ 
rence, Matloff-Nieves, & Ramos, 2013). Underfunded 
nonprofits can feel overwhelmed by the intense emphasis 


on producing “evidence-based” outcomes, especially if 
evaluation feels like an “add-on” rather than being aligned 
with and integrated into program goals. 
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This article raises up critical participatory action re¬ 
search and youth participatory evaluation as possible an¬ 
swers to this challenge. Expanding the definition of evalua¬ 
tion to include methodologies that value youth participation 
can strengthen CBOs’ capacity to create responsive out-of- 
school time (OST) programs that have meaningful impacts 
on young people’s lives. This article explores how five pro¬ 
grams use critical participatory action research and youth 
participatory evaluation to engage youth and improve pro¬ 
gram delivery. These trailblazing organizations illuminate 
the possibilities and challenges of using approaches to re¬ 
search and evaluation that reflect youth development prin¬ 
ciples and practices. 


benefits include youth leadership (Camino, 2005); strong 
youth-adult partnerships (Innovation Center for Commu¬ 
nity and Youth Development, 2005); and, according to 
some, more valid and useful research (Calvert, Zeldin, & 
Weisenbach, 2002; Golombek, 2002; Sabo-Flores, 2008). 

Involving youth in critical participatory action research 
and evaluation builds on young people’s strengths, exper¬ 
tise, and ability to create knowledge about the issues and 
programs that affect their lives. Research is conducted with 
youth, not on them. Young people are viewed as the experts 
on their own experiences. They are, in this view, completely 
capable of exploring youth issues and programs—in fact, 
they are necessary members of the 
research team. 

This perspective is remarkably 
well aligned with an assets-based 
youth development approach. The 
alignment becomes even more 
evident in the partnerships formed 
when young people and adults cre¬ 
ate research about young people’s 
programs, communities, and expe¬ 
riences. Foundational research in 
the field of youth development tells 
us that three major factors in youth 
development settings foster resil¬ 
ience and enable youth to thrive: 
caring relationships, high expecta¬ 
tions, and opportunities to contrib¬ 
ute (Bernard, 1991). A framework 
currently gaining traction in the 
field has synthesized decades of re¬ 
search evidence, practice wisdom, 
and theory to posit that children learn through develop¬ 
mental experiences that combine action and reflection, 
ideally in the context of caring, trusting relationships with 
adults (Nagaoka et al., 2015). The cycles of action and 
reflection of participatory action research, undertaken in 
respectful partnerships with adults, create ideal conditions 
for development. 

Knowledge production in partnership with young peo¬ 
ple operates at the intersection of youth development and 
youth rights (Sabo, 2003; Sabo-Flores, 2008). This cross¬ 
roads may feel quite comfortable to youth-serving organiza¬ 
tions committed to the struggle for equity on behalf of and 
in partnership with young people. However, though some 
innovators are engaging in participatory action research in 
and out of school (Cahill, 2004; Cammarota & Fine, 2008; 
Kirshner, 2006), the potential for engaging youth in par¬ 
ticipatory evaluation in OST programs is largely untapped. 


Participatory Research and 
Evaluation Approaches 

The interdisciplinary and activist 
history of critical participatory ac¬ 
tion research stretches back to Kurt 
Lewin (1946), Paulo Freire (1970), 

Orlando Fals Borda (1979), and An- 
isur Rahman (Rahman & Fals Borda, 

1991). The participatory approach 
braids critical social science, self- 
determination, and liberatory 
practice in order to interrupt injus¬ 
tice and build community capac¬ 
ity. Those who practice this youth- 
development-oriented approach bring 
to their qualitative and quantita¬ 
tive research a commitment to local 
knowledge and democratic prac¬ 
tice (Fals Borda, 1997; Torre, Fine, 

Stoudt, & Fox, 2012; Zeller-Berkman, 2014). Those who 
are affected by the topic under investigation are essential 
partners in the research process. Young people conduct¬ 
ing participatory action research in partnership with adults 
engage in ongoing and sometimes overlapping cycles of 
fact-finding, planning, action, and reflection (Lewin, 1946). 
Research teams attempt not only to understand the data, 
but also to use them to alter the underlying causes of the 
problem at hand. 

Youth participatory evaluation emerged in the late 
1990s as an extension of the field of participatory evalua¬ 
tion. Pioneers in the burgeoning field (Checkoway & Rich- 
ards-Schuster, 2003; Sabo, 2003) pushed to involve young 
people as stakeholders in program evaluations. The past 
decade has brought elaboration on how youth participatory 
evaluation happens in youth development settings and the 
benefits that occur when it does (Sabo-Flores, 2008). Such 
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Research Design 

To uncover the benefits and challenges of engaging youth in 
participatory evaluation approaches, we studied the experi¬ 
ence of staff from five CBOs who attended the five-day sum¬ 
mer Critical Participatory Action Research Institute (CPAR 
Institute) hosted by the Public Science Project in 2012. The 
Public Science Project has a 15-year history of involving 
youth as researchers, facilitating vibrant research camps and 
large-scale youth research projects on issues ranging from 
policing practices to educational equity. It acts a hub for 
scholars of critical participatory action research and a train¬ 
ing institute for those looking to implement participatory 
methods in their own contexts (Torre et al., 2012; Zeller- 
Berkman, 2014). Our five case-study CBOs all followed up 
on their learning at the institute by incorporating participa¬ 
tory evaluation in their programs. 

Of the 45 participants in the 2012 CPAR Institute, 17 
were from CBOs or university-CBO partnerships. We in¬ 
vited those who worked in OST and who wanted to en¬ 
gage youth in action research to participate in our study 
Eight staff members from five organizations agreed. The five 
CBOs varied in size, location, and program focus, as sum¬ 
marized in Table 1. 

We conducted semi-structured interviews with the 
eight CBO staff members before they participated in the 
CPAR Institute. During the institute in June 2012, we con¬ 
ducted ethnographic participant observations. Right after 
the institute, we facilitated a focus group with seven of the 
interviewees, representing all five CBOs. We conducted 


follow-up interviews three to four months after the institute, 
in fall 2012, reaching six staff members from four of the or¬ 
ganizations. Interviews and focus groups were recorded and 
then transcribed. We analyzed the data using a methodol¬ 
ogy based in grounded theory (Strauss & Corbin, 1994). 

Moving Participatory Evaluation 
from Theory to Practice 

The study participants emphasized that they brought 
youth-centered and strength-based approaches with them 
to the CPAR Institute, stressing the role of sports, the arts, 
culture, families, and civic engagement. However, only two 
of the five organizations had previously used participatory 
approaches to teaching and learning, and only one had en¬ 
gaged in participatory research. In the follow-up interviews 
a few months after the institute, all participants reported 
having used participatory strategies in program implemen¬ 
tation, design, or evaluation. 

One participant had incorporated a full participatory 
action research project into her CBO’s youth summer em¬ 
ployment program. The project engaged a team of 10 youth 
in researching young people’s experiences of schooling. The 
study participant, youth outreach coordinator at CBO 4, 
outlined the process in her follow-up interview: 

We all worked together for 25 hours a week for five 
weeks. We started off with a research camp kind of 
curriculum, combined with some curriculum on 
anti-oppression, work on sexism, racism, things like 
that.... We did school mapping .. .with some guided 


Table 1. C 

haracteristics of Case Study CBOs 


Description 

Location 

Population Served 

Staff and Partners 
Interviewed 

CBO 1 

Multiservice 

organization 

Large city in New 
York 

Low-income youth and 
their families 

Director of program 
evaluation and planning, 
evaluation specialist, 
program analyst 

CBO 2 

Arts education 

program 

Small city in 
Virginia 

Local youth from diverse 
backgrounds 

Executive director, university- 
based evaluation partner 

CBO 3 

Neighborhood- 
based organization 

Small city in 
Michigan 

Immigrant youth and their 
families 

Program director 

CBO 4 

Family education 
center 

Large city in 
Minnesota 

Immigrant African youth 

Youth outreach coordinator 

CBO 5 

Multiservice youth 

development 

organization 

Mid-sized city in 
Connecticut 

Low-income and 
immigrant youth 

Youth participatory 
action research program 
coordinator 
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questions, and one was “Where do you feel least safe or 
where do you feel most safe?” [We] prepped [research 
camp participants] a lot on interviews. They also inter¬ 
viewed each other a lot to hone in what our first round 
of interview questions would be. 

This intensive first experience 
with participatory action research 
brought both challenges and ben¬ 
efits to the organization, as we will 
discuss below. By a few months af¬ 
ter the institute, the other organiza¬ 
tions in the study had carried out 
less intensive but equally innova¬ 
tive attempts at incorporating this 
approach into their practice. Strate¬ 
gies they used with youth included 
research camps, mapping exercises, 
interviews, surveys, critical conver¬ 
sations, and performances or presen¬ 
tations of research findings by youth. 

Benefits of Participatory 
Evaluation 

The follow-up interviews revealed four benefits of engaging 
in research and evaluation processes aligned with the prin¬ 
ciples of youth development: 

• Increased youth engagement and leadership 

• Deeper adult-youth partnerships 

• Increase in participatory practices across 
the organization 

• Improved quality of the research 

Youth Engagement and Leadership 

Follow-up interviews revealed that even CBO staff 
who were already committed to youth leadership were 
impressed by the effects of critical participatory ac¬ 
tion research. They saw co-construction of knowledge 
through research as an effective way to build young 
people’s confidence. For example, the interviewee 
from CBO 5 said that the approach: 

is very effective at building leadership. My stu¬ 
dents—in particular several that had for a long 
time, as far as I can tell, been labeled “unsuccess¬ 
ful” in the classroom and schools and [were] at 
various levels of marginalization in school—really 
turned a corner.... [T]hey were able to feel suc¬ 
cessful in this learning environment we created 
together, where their knowledge, questions, and 
opinions were so valued. 


This interviewee believed that taking part in critical panici- 
patory action research in the OST program built students’ 
confidence in the academic realm as well. 

Youth-Adult Partnerships 

In follow-up interviews, study participants described how 
engaging in participatory action re¬ 
search brought changes in the dy¬ 
namics between young people and 
adults. Awareness of how adults and 
youth can share power led to more 
intentionality about who took on 
the evaluation tasks, both large and 
small—from defining a project’s re¬ 
search questions to summarizing 
the data gathered. A staff member 
from CBO 1 described how this 
new awareness informed a project 
in which a team of youth and adult 
researchers explored the meaning of 
youth success: 

We were very much focused on always being mindful 
of our relationship with the participants, and the first 
day beginning with a very broad question about what 
is research and who is a researcher.... We were very 
explicit about opportunities for participation, always 
looking for ways the young people could [participate] 
... or anything that we could do to get away from [the 
adults doing the] talking.... We had one piece where 
we had identified five subthemes of success we want¬ 
ed to zero in on, but we had a list of 20 and we gave 
everyone five stars and they voted.... We would have 
previously done show of hands, but we did it like that 
so everyone would have a voice. 

Study respondents spoke about how engaging youth 
in participatory evaluation enabled them not only to relin¬ 
quish control, but also to collaborate with young people 
and engage them as both teachers and learners. Some par¬ 
ticipants, including the program director of CBO 3, said that 
the CPAR Institute enhanced their commitment to view¬ 
ing young people as assets: “[The Institute] for me has... 
enhanced my belief [that youth] are a source of amazing in¬ 
formation and that, when we listen, we find out so much.” 

For program directors, working as full partners with 
youth and their communities involved questioning their 
traditional approach to building “clear boundaries” be¬ 
tween staff and community members. As one respondent 
from CBO 1 put it, a participatory approach can clash with 
the traditional notion that “staff to have very clear bound- 


Awareness of how adults 
and youth can share 
power led to more 
intentionality about who 
took on the evaluation 
tasks, both large and 
small—from defining a 
project's research 
questions to summarizing 
the data gathered. 
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aries so they are not friends, they don’t fraternize.” In the 
focus group, several staff members agreed that boundaries 
can serve as a means of demonstrating “who is in charge” in 
a youth program. However, they also agreed that boundar¬ 
ies helped staff members feel safe in working with youth 
and their communities. Organizations that incorporate par¬ 
ticipatory evaluation may need to reflect on ways to balance 
the need for healthy boundaries with the need for open 
communication and mutual trust. 

Participatory Approaches 
Across the Organization 

A third theme in the interviews was that participatory ap¬ 
proaches offered benefits not just for the OST program and 
its youth and staff but for the entire CBO. Even when the 
task at hand was not research, respondents said, they had 
become more comfortable with letting young people take 
the lead. Participatory practices and sharing leadership with 
young people was described by one participant as a “PAR- 
esque” approach that was seeping into his CBOs culture. 

The evaluation director of CBO 1 reported that having 
integrated youth into critical participatory action research 
was affecting work with the staff: 

We introduced icebreakers into program meetings, just 
to chill people out. And then we realized that the ice¬ 
breakers we were using were really about establishing 
common ground, so that we would, for instance, have 
a meeting with the afterschool staff, and the icebreaker 
was “Tell us about your first involvement with after- 
school”. . .So we all kind of established our stake and 
that we were all stakeholders in afterschool programs 
with a lot of commitment to them and perspective. 
[W]e really have developed this 
process in these meetings about 
power relations and establish¬ 
ing common ground and com¬ 
mon purpose. 

Organizations that incorporate 
a participatory frame into youth- 
centered and strengths-based ap¬ 
proaches may experience benefits 
across the entire organization, not 
just with the youth. 

Quality of the Research 

A fourth benefit the CBO respondents noted was that the 
quality of their research improved. CBO staff were com¬ 
mitted to participatory practices not only out of idealism, 
but also because these practices better equipped them to 


carry out valid research. One respondent mentioned that 
collaboration with youth on an evaluation survey brought 
up issues “that would have never come to mind” for the 
adult staff members. The program coordinator from CBO 
5 put it this way: 

A PAR approach has definitely taught me that people 
who are “the subjects” of the research need to be in 
the room from the first, including designing what the 
research questions have to be. I learned that really early 
on.. .when we interviewed youth to hire them and we 
created our questions about school.. .And they all talk¬ 
ed about favoritism. And that, to me, was a great les¬ 
son, because if I had designed the interview questions 
about youth experience, [I] never would have asked 
about favoritism. 

Challenges of Participatory Evaluation 

In addition to benefits, the follow-up interviews revealed 
challenges in involving youth in participatory action re¬ 
search and evaluation. One major challenge is that these 
approaches take time. One CBO staff member articulated 
a common issue: feeling torn between being realistic 
about the workload and being committed to a participa¬ 
tory approach. 

1 am very happy with the way [the project] turned out, 
but it was also a reality check, because it took a lot of 
our time. And 1 am here thinking I would not want 
to do this again until next summer because I have so 
many other projects on my plate. 

The youth outreach coordinator from CBO 4 echoed 
this sentiment, explaining that the budget and design of her 
program did not allow for the level of 
youth participation that would have 
produced high-quality data. The five 
weeks allotted for research did not al¬ 
low the youth to take part in designing 
data collection instruments, conduct¬ 
ing the research, and analyzing the 
data. This staff member struggled with 
how much she and the other facilita¬ 
tors should structure the work ahead 
of time and how much to leave open 
for the adult-youth team to shape to¬ 
gether. She compromised by starting 
the process with a well-defined topic for the project and with 
structured workshops that helped the research team come 
alive. Once the team had agreed on a subtopic and method for 
the projects, she provided scaffolding and assistance to help the 
youth complete their goals in the available time. 


CBO staff were committed 
to participatory practices 
not only out of idealism, 
but also because these 
practices better equipped 
them to carry out valid 
research. 
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A second challenge was lack of institutionalization 
of participatory approaches to program design and eval¬ 
uation. The executive director of 
CBO 2 explained: 

I definitely feel reluctant to 
our kids having to fill out tons 
of tests like rats in a maze and 
put them through pre- and 
post-tests. Honestly, we run on 
an extremely skinny budget, 
and we don’t have the admin¬ 
istrative capacity to administer 
pre- and post-tests or evaluate 
them or administer the data.... 

Not to say we don’t want to 
demonstrate the impact of our 
program to people, but I am 
just concerned that funders and 
foundations are going over the 
top in creating really unrealis¬ 
tic requirements [for organiza¬ 
tions] such as ours, which will 
be at risk of going out of busi¬ 
ness because of these require¬ 
ments. And I think CPAR can 
perhaps provide tools that are 
more user-friendly and friendly 
to the population and that are not viewed punitively 

Clearly this interviewee understands the importance of 
evaluations that demonstrate program impact. At the same 
time, the comments reflect a feeling shared by other interview¬ 
ees that certain approaches to evaluation have negative con¬ 
notations for CBO staff. This executive director articulates the 
possibility that youth critical action research can contribute to 
evaluation that is “more user-friendly” and that, rather than 
punishing CBOs through funding cuts, promotes a culture of 
accountability and constant improvement. 

Interviewees explained that the transition from providing 
a one-off participatory project or class to making participatory 
evaluation a permanent fixture in the organization was hard. 
Surprisingly, the interviews revealed hopefulness about the 
coexistence of outcomes-driven evaluation and critical partici¬ 
patory action research. Respondents felt that their CBOs and 
funders might be more open than they had thought to partici¬ 
patory program design and evaluation. 

Evaluation Aligned with Program Goals 

The New York-based multiservice organization whose eval¬ 
uation staff attended the CPAR Institute saw its evaluation 


culture positively affected by the inclusion of youth per¬ 
spectives. One benefit reported by this organization’s study 
participants was that program staff 
took a more active role in the design 
of evaluation strategies, rather than 
viewing the evaluation staff as the 
sole experts. As a result, the evalua¬ 
tion process was enriched by exper¬ 
tise of staff who knew the day-to-day 
operation of the programs and who 
had direct contact with youth. 

A conversation among focus 
group participants echoed the idea 
that using critical participatory ac¬ 
tion research shifted the culture of 
evaluation in their organizations: 
Participant A: It certainly provided 
a whole new avenue for how we 
can make [the evaluation] process 
more friendly to the participants 
and align ourselves more with 
them in ways that engage them 
and ... bring them into a process 
that demonstrates to them the ad¬ 
ditional talents they have to help 
provide insight into why or why 
not the program is working and 
improve it.... I think [PAR is] a much improved way 
of trying to help the entire situation of having to do so 
much more evaluation these days. 

Participant B: I think I am very used to the scientific 
method approach where you go in with a hypothesis. 
So doing research this way is kind of foreign to me. 
PAR has made it clear—it is a much more valid form. I 
always thought so, but until you really see it and really 
learn about it, it is kind of foreign. 

Participant A: [The Institute] has helped me to see that 
[evaluation] can be a very empowering tool versus a 
very overpowering or dominating, exploitative tool. 

This dialogue envisions a scenario in which afterschool 
program evaluation can not only account for outcomes such 
as credits gained, but also create space for youth action re¬ 
search projects that influence people and programs. In this 
youth development approach to evaluation and research, 
study participants saw a tool that could both build young 
people’s talents and reveal insights to enable program im¬ 
provement. 

Our study suggests that, in order to experience these 
benefits, CBOs need to provide institutional support for 
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participatory approaches to design and evaluation. Staff 
also need to identify the spaces in the organization and 
its programs where such approaches will be a good fit. 
Staff from both of the sites that had finished action re¬ 
search projects at the time of the follow-up interview said 
that their executive directors were open to and supported 
participatory evaluation. A staffer from CBO 1 described 
how one program in the organization was open to par¬ 
ticipatory research while another was rigidly bound to a 
different approach to evaluation: 

The project in the Bronx received lots of support 
from the highest levels here. This was included in 
a packet to one of our major funders this morning, 
and they were very happy with our organization for 
promoting youth voice.... On the other hand, we 
have a lot of pressures going on right now with our 
child welfare program and evidence-based models. 

CBO 2, the other site that had completed a youth 
action research project, also reported that the work was 
“pretty well received” in the city’s youth affairs agency. 
This staff member stated that the project “brought a 
louder voice back to Youth Affairs about the necessity 
of having more youth involvement at every layer of the 
organization, having more youth involved in planning 
our programs.” This respondent expressed some 
frustration that grant applications reinforce top-down 
hierarchies in youth-adult relationships by, for example, 
not allowing applicants to identify young people simply 
as “co-researchers.” However, this respondent said, “The 
foundation we are applying to thinks differently about, 
and is open in their perspective on, hierarchies in youth- 
adult collaborations.” 

The CBO program and evaluation staff in our study 
saw critical participatory action research as a useful and 
valid tool. In a funder climate that emphasizes evalua¬ 
tion, the alignment of participatory research with an 
assets-based approach seems to be attractive to executive 
directors and evaluation staff who are looking to produce 
useful and valid data while also developing capacities 
among staff members and youth. Unlike evaluation pro¬ 
cesses that are perceived as add-ons or resource drains, 
youth participatory action research adds value by align¬ 
ing with and expanding on program goals. 

Unleashing a Virtuous Cycle 

The youth programs featured in this article highlight the 
power and potential of using research and evaluation de¬ 
signs that are aligned with positive youth development. 
These sites have found that involving youth in critical 


participatory action research can create valid data to 
drive programs while promoting practices that youth and 
adults find “user-friendly” and “empowering.” Participa¬ 
tory approaches offer CBOs a way to develop research 
about youth programs that is driven by the youth and 
communities who are most affected. 

While it is not without challenges, participatory 
action research offers benefits including increased 
youth engagement and leadership, deeper adult-youth 
partnerships, an increase in participatory practices across 
the organization, and greater validity in the research 
instruments and analyses used for evaluation. These 
benefits reinforce conditions that enable young people 
to thrive: partnerships with adults characterized by 
caring and trusting relationships, high expectations, and 
multiple opportunities for both generations to contribute 
to cycles of reflection and action. The study thus suggests 
that using an evaluation framework that is aligned with 
the principles of youth development unleashes a virtuous 
cycle: The evaluative process supports the very outcomes 
youth development programs are designed to achieve. 
Though our findings hint at the existence of this virtuous 
cycle, its process and its implications for program design, 
implementation, and evaluation must be revealed by 
further research. 

To unleash this virtuous cycle more often, funders 
need to make an explicit commitment to a youth 
development approach to research and evaluation. Our 
interviewees said that their funders and administrators 
expressed interest in and support for youth involvement 
in research and evaluation. Though this finding is 
promising, funders and leaders still need to let youth 
program staff know that participatory approaches are not 
only permitted, but valued. Programs need additional 
funding to support the time and effort it takes to carry 
out participatory evaluation driven by deep youth- 
adult partnerships. Similarly, capacity-building support 
is necessary if our field is to shift the current culture of 
evaluation to one better aligned with youth development 
principles and practices. 

Increasing funding and building capacity for youth 
participation in action research will help to institutionalize 
evaluation approaches aligned with youth development. 
Capitalizing on these approaches could prove to be a win- 
win scenario for funders and youth programs who are 
striving to maximize their impact, shrink the pervasive 
opportunity gap, and increase youth engagement every 
step of the way. 
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